Multi-Attribute Monitoring for Anomaly Detection: a Reinforcement Learning Approach based on Unsupervised Reward

International audienceThis paper proposes a new method to solve the monitoring and anomaly detection problems of Low-power Internet of Things (IoT) devices. However, their performances are constrained by limited processing, memory, and communication, usually using battery-powered energy. Polling driven mechanisms for monitoring the security, performance, and quality of service of these networks should be efficient and with low overhead, which makes it particularly challenging. The present work proposes the design of a novel method based on a Deep Reinforcement Learning (DRL) algorithm coupled with an Unsupervised Learning reward technique to build a pooling monitoring of IoT networks. This combination makes the network more secure and optimizes predictions of the DRL agent in adaptive environments

Reinforcement and deep reinforcement learning for wireless Internet of Things: A survey

International audienceNowadays, many research studies and industrial investigations have allowed the integration of the Internet of Things (IoT) in current and future networking applications by deploying a diversity of wireless-enabled devices ranging from smartphones, wearables, to sensors, drones, and connected vehicles. The growing number of IoT devices, the increasing complexity of IoT systems, and the large volume of generated data have made the monitoring and management of these networks extremely difficult. Numerous research papers have applied Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) techniques to overcome these difficulties by building IoT systems with effective and dynamic decision-making mechanisms, dealing with incomplete information related to their environments. The paper first reviews pre-existing surveys covering the application of RL and DRL techniques in IoT communication technologies and networking. The paper then analyzes the research papers that apply these techniques in wireless IoT to resolve issues related to routing, scheduling, resource allocation, dynamic spectrum access, energy, mobility, and caching. Finally, a discussion of the proposed approaches and their limits is followed by the identification of open issues to establish grounds for future research directions proposal

Leveraging Reinforcement Learning for Adaptive Monitoring of Low-Power IoT Networks

International audienceLow-power Internet of Things (IoT) networks are widely deployed in various environments with resource constrained devices, making their states monitoring particularly challenging. In this paper, we propose an adaptive monitoring mechanism for low-power IoT devices, by using a reinforcement learning (RL) method to automatically adapt the polling frequencies of the collected attributes. Our goal is to minimize the number of monitoring packets while keeping accurate and timely detection of threshold crossings associated to supervised attributes. We study the various RL parameter settings under different monitoring attribute behaviors using OpenAi Gym simulator. We implement the RL based adaptive polling in Contiki OS and we evaluate its performance using Cooja simulator. Our results show that our approach converges to optimal polling frequencies and outperforms static periodic notification-based methods by reducing the number of monitoring packets, with a percentage of correctly detected threshold crossings exceeding 80%

A non-cooperative game-theoretic framework for resource allocation in network virtualization

International audienceNetwork virtualization is a new technology thataimsatallowingmultiplevirtualnetworks(VNs)tocoexistinthe same equipment and to hide the heterogeneity of networkinfrastructure. The critical issue for a given infrastructureprovider (InP), is how to provide customized and on demandresources for multiple service providers (SPs) with differentQualityofService(QoS)requirements.Theshouldalsofairlydistribute the network physical resources, such as bandwidthof each physical link, buffer spaces, and processing cycles ateach node. In this paper, we propose a new framework basedon game theory, for both link and node dynamic allocationbetween multiple infrastructure providers (InPs) and serviceproviders (SPs). Our approach focuses on provisioning andmanaging the physical resources in a virtualized networkinfrastructure. We propose a two-stage approach based onnon-cooperative games. The first one is the resource negotiation game where the SP requests link and node resourcesfrom multiple InPs. The InP may reject the SP’s requestwhen it can potentially cause network congestion. The second stage of the proposal concerns dynamic resource provisioning and consists of two non cooperative games; the node allocation game and the link allocation game. Theobjective of both games is to allocate physical resources fordifferent isolated VNs that are sharing the same physicalsubstrate network. In the node allocation game, the proportional share mechanism is used. Every SP assigns a weightand submits a bid to each physical node and thereafter itreceives a share proportional to its bid. In the link allocation game we investigate the case when multiple SPs compete for a portion of the available physical network capacity.Simulation results show that the proposed approach achieveshigh resource utilization, improves the network performance,and fairly distributes the link and node resources betweenmultiple SPs